Persuasion modeling is a key building block for conversational agents. Existing works in this direction are limited to analyzing textual dialogue corpus. We argue that visual signals also play an important role in understanding human persuasive behaviors. In this paper, we introduce the first multimodal dataset for modeling persuasion behaviors. Our dataset includes 199 dialogue transcriptions and videos captured in a multi-player social deduction game setting, 26,647 utterance level annotations of persuasion strategy, and game level annotations of deduction game outcomes. We provide extensive experiments to show how dialogue context and visual signals benefit persuasion strategy prediction. We also explore the generalization ability of language models for persuasion modeling and the role of persuasion strategies in predicting social deduction game outcomes. Our dataset, code, and models can be found at https://persuasion-deductiongame.socialai-data.org.

Image-based head swapping task aims to stitch a source head to another source body flawlessly. This seldom-studied task faces two major challenges: 1) Preserving the head and body from various sources while generating a seamless transition region. 2) No paired head swapping dataset and benchmark so far. In this paper, we propose an image-based head swapping framework (HS-Diffusion) which consists of a semantic-guided latent diffusion model (SG-LDM) and a semantic layout generator. We blend the semantic layouts of source head and source body, and then inpaint the transition region by the semantic layout generator, achieving a coarse-grained head swapping. SG-LDM can further implement fine-grained head swapping with the blended layout as condition by a progressive fusion process, while preserving source head and source body with high-quality reconstruction. To this end, we design a head-cover augmentation strategy for training and a neck alignment trick for geometric realism. Importantly, we construct a new image-based head swapping benchmark and propose two tailor-designed metrics (Mask-FID and Focal-FID). Extensive experiments demonstrate the superiority of our framework. The code will be available: https://github.com/qinghew/HS-Diffusion.

Point cloud registration (PCR) is a popular research topic in computer vision. Recently, the registration method in an evolutionary way has received continuous attention because of its robustness to the initial pose and flexibility in objective function design. However, most evolving registration methods cannot tackle the local optimum well and they have rarely investigated the success ratio, which implies the probability of not falling into local optima and is closely related to the practicality of the algorithm. Evolutionary multi-task optimization (EMTO) is a widely used paradigm, which can boost exploration capability through knowledge transfer among related tasks. Inspired by this concept, this study proposes a novel evolving registration algorithm via EMTO, where the multi-task configuration is based on the idea of solution space cutting. Concretely, one task searching in cut space assists another task with complex function landscape in escaping from local optima and enhancing successful registration ratio. To reduce unnecessary computational cost, a sparse-to-dense strategy is proposed. In addition, a novel fitness function robust to various overlap rates as well as a problem-specific metric of computational cost is introduced. Compared with 7 evolving registration approaches and 4 traditional registration approaches on the object-scale and scene-scale registration datasets, experimental results demonstrate that the proposed method has superior performances in terms of precision and tackling local optima.

This technical report briefly describes our JDExplore d-team's Vega v2 submission on the SuperGLUE leaderboard. SuperGLUE is more challenging than the widely used general language understanding evaluation (GLUE) benchmark, containing eight difficult language understanding tasks, including question answering, natural language inference, word sense disambiguation, coreference resolution, and reasoning. [Method] Instead of arbitrarily increasing the size of a pretrained language model (PLM), our aim is to 1) fully extract knowledge from the input pretraining data given a certain parameter budget, e.g., 6B, and 2) effectively transfer this knowledge to downstream tasks. To achieve goal 1), we propose self-evolution learning for PLMs to wisely predict the informative tokens that should be masked, and supervise the masked language modeling (MLM) process with rectified smooth labels. For goal 2), we leverage the prompt transfer technique to improve the low-resource tasks by transferring the knowledge from the foundation model and related downstream tasks to the target task. [Results] According to our submission record (Oct. 2022), with our optimized pretraining and fine-tuning strategies, our 6B Vega method achieved new state-of-the-art performance on 4/8 tasks, sitting atop the SuperGLUE leaderboard on Oct. 8, 2022, with an average score of 91.3.

Bimanual activities like coffee stirring, which require coordination of dual arms, are common in daily life and intractable to learn by robots. Adopting reinforcement learning to learn these tasks is a promising topic since it enables the robot to explore how dual arms coordinate together to accomplish the same task. However, this field has two main challenges: coordination mechanism and long-horizon task decomposition. Therefore, we propose the Mixline method to learn sub-tasks separately via the online algorithm and then compose them together based on the generated data through the offline algorithm. We constructed a learning environment based on the GPU-accelerated Isaac Gym. In our work, the bimanual robot successfully learned to grasp, hold and lift the spoon and cup, insert them together and stir the coffee. The proposed method has the potential to be extended to other long-horizon bimanual tasks.

现实世界中的数据通常显示出长尾巴的开放式（带有看不见的类）分布。实践识别系统必须在多数（头）和少数族裔（尾巴）阶级之间取得平衡，在整个分布中进行概括，并承认新颖的阶级（公开阶级）。我们将开放的长尾识别++（OLTR ++）定义为从这种自然分布的数据中学习，并优化了包括已知和开放类的平衡测试集的分类精度。 OLTR ++在一种集成算法中处理不平衡的分类，很少的学习，开放式识别和积极学习，而现有的分类方法通常仅着眼于一个或两个方面，并且在整个频谱中交付不佳。关键挑战是：1）如何在头和尾巴之间共享视觉知识，2）如何减少尾巴和开放式阶级之间的混淆，以及3）如何用学习知识积极地探索开放的课程。我们的算法OLTR ++将图像映射到特征空间，以便视觉概念可以通过记忆关联机制和学识渊博的指标（动态元元素）相互关联，这两者都尊重所封闭的见解类别的封闭世界分类并承认的新颖性打开课程。此外，我们提出了一个基于视觉记忆的主动学习方案，该方案学会以数据效率的方式识别未来扩展的开放类。在三个大规模开放的长尾数据集中，我们从Imagenet（以对象为中心），位置（以场景为中心）和MS1M（面部为中心）数据策划了三个标准基准（CIFAR-10-LT，CIFAR，CIFAR，CIFAR） -100-LT和Inaturalist-18），我们作为统一框架的方法始终展示竞争性能。值得注意的是，我们的方法还显示出积极探索开放阶级和对少数群体的公平分析的强大潜力。

在本文中，我们提出了第一个基于变压器的模型，该模型解决了以自我为中心凝视估计的具有挑战性的问题。我们观察到，全局场景上下文和本地视觉信息之间的连接对于从以自我为中心的视频帧进行凝视固定至关重要。为此，我们设计了变压器编码器将全局上下文嵌入为一个附加的视觉令牌，并进一步提出了一种新型的全球 - 本地相关（GLC）模块，以明确模拟全局令牌和每个本地令牌的相关性。我们在两个以自我为中心的视频数据集中验证了我们的模型-EGTEA凝视+和EGO4D。我们的详细消融研究证明了我们方法的好处。此外，我们的方法超过了先前的最新空间。我们还提供了其他可视化，以支持我们的主张，即全球 - 本地相关性是预测以自我为中心视频的凝视固定的关键表示。更多详细信息可以在我们的网站（https://bolinlai.github.io/glc-egogazeest）中找到。

现有的伪造检测方法通常将面部伪造视为二进制分类问题，并采用深层卷积神经网络来学习歧视性特征。理想的判别特征应仅与面部图像的真实/假标签有关。但是，我们观察到，香草分类网络学到的特征与不必要的属性（例如伪造方法和面部身份）相关。这种现象将限制伪造的检测性能，尤其是对于概括能力。在此激励的基础上，我们提出了一种新型方法，该方法利用对抗性学习来消除不同伪造方法和面部身份的负面影响，该方法有助于分类网络学习固有的常见歧视性特征，以进行伪造伪造。为了利用缺乏面部身份的地面真实标签的数据，我们根据来自现成的面部识别模型得出的相似性信息设计了一个特殊的身份歧视器。在对抗性学习的帮助下，我们的伪造检测模型学会了通过消除伪造方法和面部身份的影响来提取共同的歧视特征。广泛的实验证明了该方法在数据集内和交叉数据集评估设置下的有效性。

联合学习（FL）可以培训全球模型，而无需共享存储在多个设备上的分散的原始数据以保护数据隐私。由于设备的能力多样化，FL框架难以解决Straggler效应和过时模型的问题。此外，数据异质性在FL训练过程中会导致全球模型的严重准确性降解。为了解决上述问题，我们提出了一个层次同步FL框架，即Fedhisyn。 Fedhisyn首先根据其计算能力将所有可​​用的设备簇分为少数类别。经过一定的本地培训间隔后，将不同类别培训的模型同时上传到中央服务器。在单个类别中，设备根据环形拓扑会相互传达局部更新的模型权重。随着环形拓扑中训练的效率更喜欢具有均匀资源的设备，基于计算能力的分类减轻了Straggler效应的影响。此外，多个类别的同步更新与单个类别中的设备通信的组合有助于解决数据异质性问题，同时达到高精度。我们评估了基于MNIST，EMNIST，CIFAR10和CIFAR100数据集的提议框架以及设备的不同异质设置。实验结果表明，在训练准确性和效率方面，Fedhisyn的表现优于六种基线方法，例如FedAvg，脚手架和Fedat。

图像DeBlurring旨在恢复模糊图像中的详细纹理信息或结构，这已成为许多计算机视觉任务中必不可少的一步。尽管已经提出了各种方法来处理图像去除问题，但大多数方法将模糊图像视为一个整体，并忽略了不同图像频率的特征。在本文中，我们提出了一种新方法，称为图像脱毛的多尺度频率分离网络（MSFS-NET）。 MSFS-NET将频率分离模块（FSM）引入编码器 - 模块网络体系结构中，以在多个尺度上捕获图像的低频和高频信息。然后，分别设计了一个循环一致性策略和对比度学习模块（CLM），以保留低频信息，并在Deblurring期间恢复高频信息。最后，不同量表的特征是通过跨尺度特征融合模块（CSFFM）融合的。基准数据集的广泛实验表明，所提出的网络可实现最先进的性能。